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Introduction 


e Daniel van Eeden 
ወ PingCAP is the company behind TIDB 
© Founded in 2015 
O Offices all around the world 
© Customers all over the world 
O Open Source Culture 


፪ (D TiDB 
Basics 


The TIDB database platform is an opensource MySQL compatible database 
System. 


By re-thinking and re-writing the database with scalability in mind PingCAP 
created a database that is easily scalable and has high availability builtin. 
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The problems with MySQL 


MySQL was created around 1995 

...for computer systems that were available at the time. 
...for disks that were available at the time. 

...for (inter)networks that were available at the time. 


[7 [— | መጨ - 
ሽ | © C) 


TURENE TER à 
PORTATA 
ΠΠ ERR 3 NN 


MEM Φτιρβ 
High availability with MySQL 


An often-used setup with MySQL is to have a primary and multiple replicas. 
Then if the primary fails you promote a replica. 


Replica promotion is not automated. 
Is your replica up to date? 
Use a loadbalancer for service discovery? 


How to fail back once the primary is back? ሥ፦. 
There is no leadership election. 
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PD cluster 


High availability with TI 


All components are redundant. 
The application is expected to retry failed 
connections. 
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Tables are split into smaller parts of DistSQL 
: T : T API 
around 96 MiB each. These "data regions ET ..- , 
are stored on TIKV. 25 
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Placement Driver (PD) 
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The placement driver: ... 
e!s a Raft group itself nen ne eee ene eee eee eee! 
ወ Has etcd embedded TSO/Data location | 
eis the Time Stamp Oracle (TSO): it gives 
out timestamps. These timestamps are 
used in transactions and in the MVCC 
System 
@ Takes care of data placement: 
ርን TIKV servers have labels 
O labels are used to ensure a raft group 
spans multiple availability zones 
OSplitting big data regions, merging 
small data regions, splitting hot data 
regions, evenly distributing regions 
across the cluster. 
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TIDB Server is one of the components of "ul 
the TIDB Platform. This is abit confusing. (0 . . ሥሥ. 


TIDB Server 


TIDB Server is written in Go and doesn't 
share any code with MySQL. 
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This implements the MySQL protocol and 
syntax. 
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TIDB ts stateless, it doesn't store data. 
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TIKV is a key-value store. This is a CNCF "ul 
project. ο ο. 


τικν 


Database tables are stored with a RowID 
or PK as key and the columns as values. 


DistSQL 
A database table is split up into multiple re MAN 
data regions. Each data region Is a raft ፳፻ 
group of (by default) three nodes. $8 
2$ 
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TiKV node TiKV node TiKV node 
SELECT id FROM E i 
orders WHERE 


id=1000001 
sels TiDB node - 


TiDB node 


TiKV node TiKV node TiKV node 
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Scalability with MySQL 


With a MySQL all writes go to the primary. 
All nodes store a complete copy of the data. 


Scalability with MySQL 


Scaling reads? 
Add more re plicas 


Scalability with MySQL 


Scaling writes? 
Replace the primary with a bigger machine 


Scalability with MySQL 


Scaling data volume? 
Add bigger disks 


Scalability with MySQL 


Need to scale more? 
Shard on the application side 
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PD cluster 


Scalability with TIDB 


Both reads and writes can go to any of the TIDB nodes. 
Scaling reads? ul . - ፐ.. 
Add more nodes 
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Scaling writes? 


DistSQL 
API 
Add more nodes = 


Scaling data volume? 
Add more nodes 


Application via 
MySQL protocol 
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Need to scale more? 
Add more nodes 


TiKV node TiKV node TiKV node 
SELECT id FROM E i 
orders WHERE 


id=1000001 
sels TiDB node - 


TiDB node 
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TIKV node TIKV node TIKV node 
SELECT id FROM ፪ 
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orders WHERE TiDB node E m 
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TIKV node 


SELECT id FROM Lave 
orders WHERE TiDB node 
id=1000001 1 
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TiDB node "T 
TiDB node E 
TiDB node | 
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Reports ) 


Analytics and ETL 


Some databases are built for transactional workloads 
(OLTP) 

Some databases are built for analytical workloads 
Extract-Iransform-Load (ETL) is often done once a day 
Reports might be based on hours old data 


ETL 
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Analytics and ETL 


e Replace both OLTP and OLAP systems with TIDB 
e No more ETL required Reports 


e No more outdated reports 
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Analytics and ETL 


TiFlash stores one or more copies of selected tables 
in row based format. The primary copy Is always on 
TIKV. 

TiFlash is based on Clickhouse. 

TiFlash Joins the raft groups as a learner. It will never 
become a leader. 

To use TiFlash you set the number of copies on a per 
table level. Multiple copies are good for redundancy 
and allow parallel execution of queries. 

The optimizer is smart enough to select the row store 
(TIKV) or the column store (TiFlash) based on the type 
of query. 

Execution of a query can use both TIKV and TiFlash 
for executing different parts of the query. 

Transaction isolation is guaranteed, also when TiFlash 
Is used. 
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Batteries included: BR 


External 
storage 
Backup and Restore (BR) == 


The TIKV servers all write to 
shared storage to create a 
backup. 

The TIKV servers all read from 
shared storage to restore. 
This makes backups fast and 


Scalable. 
External storage could be S3, ΘΠ 


Backup cluster 


MinlO, GCS or a shared 
filesystem. 
Opensource. br backup xxx 
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External 


storage 


Restore cluster 


EN EN 
A 


br restore xxx : 


፪ ፪ (D TiDB 
Batteries included 


e Dumpling 

O Dumping data to SQL or CSV in parallel. Can also dump data from MySQL. 
e Lightning 

O Restore a dump that was made with dumpling. 

O Can import CSV made by other tools. 

O Can import directly to TIKV, bypassing the SQL layer. 

OCan also Import via SQL statements to TIDB. 
e Main usecase is to migrate data from MySQL 
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Batteries included 


Data Migration (DM) 


9 ነ አዘው duta fron MySQL 5. ο. 
to TiDB. Comband ፪ሽ 
ወ UsesDumpling/Lightningto ΠΝ ο 
copy the initial copy. ፪ > 
e Note that this is also madeto - 
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be high available. 


MySQL/MariaDB MySQL/MariaDB 
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Batteries included 


TICDC 
e Change Data Capture 
ወ Send events to Kafka, MySQL or another TIDB Cluster. 
e Also high available 
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br backup xxx 


Backup cluster 
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External 
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Restore cluster 
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br restore xxx 


Leave your feedback! 


You can rate the talk 
and give feedback on 
what you've liked or 
what could be 
improved 


Co-organizer 


Yandex 


